Apparatus for optimizing inspection of exterior of target object and method thereof

ABSTRACT

There is provided a technique that includes: a camera configured to capture images of the target object; a memory configured to store the images of the target object and feature data including one or more predetermined exterior features of the target object; and a processor configured to: determine a first process configuration for an operation including a plurality of image processes; perform the operation under the first process configuration; generate inspection data from the sets of images that have been processed; generate an inspection score by comparing the inspection data with the feature data; compare the inspection score with a predetermined threshold score; set the first process configuration as an optimal configuration if the inspection score satisfies the predetermined threshold score.

TECHNICAL FIELD

This application is based upon and claims the benefit of priority fromU.S. Provisional Patent Application No. 62/529,582, filed on Jul. 7,2017 the entire contents of which are incorporated herein by reference.

The present disclosure relates to an apparatus for optimizing anoperation for processing a plurality of image streams and a methodthereof.

BACKGROUND ART

In connection with performing MOI (Mechanical Optical Inspection), it ispossible to initially obtain an image of a target object, which is asubject of the MOI, and then to determine whether the target objectconforms to the specification on the basis of the image. In other words,in the process of inspecting the target object, a diagnosis may beconducted, for example, in connection with whether the correspondingtarget object has a shape conforming to the specification and/or whetherthe surface of the target object has a defect.

However, it may be difficult to accurately specify a defect of thetarget object. To this end, various image processes or image processingtechniques may be performed in connection with the image of the targetobject. There may be various types of image processes for sensing adefect, at least because there may be no clear definition of a defect ofthe target object, and there are various types of defects.

In order to sense various types of defects, a MOI operation may requirean appropriate configuration. In other words, in connection withperforming a MOI operation, configurations may be required in connectionwith at least one of the following examples: to what image, among imagesof the target object, image processes are to be applied; what kind ofimage process is to be included in the MOI operation; in what sequencerespective image processes are to be applied to the image; and howparameter values used for respective image processes are to be selected.

Such a configuration process has a problem in that a significant amountof time may be required. Furthermore, according to this scheme, aconfiguration corresponding to a local maxima or local minima may beerroneously derived as the optimal configuration in the process ofapplying a certain configuration to the MOI operation.

DISCLOSURE OF INVENTION Technical Problem

The present disclosure, in order to solve the above-mentioned problemsof the related art, provides a technology that optimizes a process forprocessing one or more sets of images in order to inspect a defect of atarget object.

Solution to Problem

According to one embodiment of the present disclosure, there is provideda technique that includes: at least one camera configured to capture oneor more sets of images of the target object; a memory configured tostore the one or more sets of images of the target object and featuredata including one or more predetermined exterior features of the targetobject; and a processor configured to: obtain the one or more sets ofimages from the memory, each set of images being captured under one ormore predetermined settings; determine a first process configuration foran operation including a plurality of image processes; perform theoperation on the one or more sets of images under the first processconfiguration; generate inspection data on the exterior of the targetobject from the one or more sets of images that have been processedunder the first process configuration; generate an inspection score bycomparing the inspection data on the exterior of the target object withthe feature data; compare the inspection score with a predeterminedthreshold score; set the first process configuration as an optimalconfiguration if the inspection score satisfies the predeterminedthreshold score; and determine a second process configuration for theoperation if the inspection score does not satisfy the predeterminedthreshold score.

Specifically, the processor, when determining the first processconfiguration, is further configured to: determine a first set of imagesfrom the one or more sets of images, on which the operation under thefirst process configuration is performed; determine a first set of imageprocesses from the plurality of image processes, which is performed inthe operation under the first process configuration; determine firstparameter values for the first set of image processes; and determine afirst sequence of the first set of image processes.

Specifically, the processor, when generating the inspection score, isfurther configured to: obtain one or more flags from the inspectiondata, each of the one or more flags indicating whether the exterior ofthe target object includes at least one of the predetermined exteriorfeatures; determine whether each of the one or more flags is consistentwith the feature data stored in the memory; and generate the inspectionscore that indicates a ratio of a number of flags that are consistentwith the feature data to a total number of the one or more flags.

Specifically, the processor, when generating the inspection score, isfurther configured to: obtain a first area on the target object from theinspection data, the first area including at least one of thepredetermined exterior features; obtain a second area on the targetobject from the feature data pre-stored in the memory, the second areaincluding the at least one of the predetermined exterior features;determine an overlapping area between the first area and the secondarea; and generate the inspection score that indicates a proportion ofthe overlapping area in the second area.

Specifically, the processor, when determining the second processconfiguration, is further configured to determine a second set of imagesfrom the one or more sets of images, a second set of image processesfrom the plurality of image processes, second parameter values for thesecond set of image processes, and a second sequence of the second setof image processes, according to a difference between the inspectionscore and the predetermined threshold score.

Specifically, the processor, when determining the second processconfiguration, is further configured to: derive a difference between aprevious inspection score, which has been generated using a previousprocess configuration for the operation before performing the operationunder the first process configuration, and the inspection scoregenerated using the first process configuration for the operation; anddetermine the second process configuration by modifying the firstprocess configuration according to the derived difference.

Specifically, the processor is further configured to perform theoperation under the second process configuration.

Specifically, an apparatus for optimizing an inspection of an exteriorof a target object further comprises: a light source configured toirradiate light to the target object, wherein the one or morepredetermined settings include at least one of a wavelength of the lightirradiated to the target object, an irradiation angle of the light intothe target object, an illumination of the light, and a type of the atleast one camera.

Specifically, the first sequence includes a sub-sequence that applies atleast two different image processes, among the first set of imageprocesses, to the first set of images in parallel.

Specifically, the first sequence includes a sub-sequence that applies asingle image process, among the first set of image processes, to thefirst set of images after applying the at least two different imageprocesses to the first set of images in parallel.

According to one embodiment of the present disclosure, there is provideda technique that includes: capturing one or more sets of images of thetarget object, each set of images being captured under one or morepredetermined settings; determining a first process configuration for anoperation including a plurality of image processes; performing theoperation on the one or more sets of images under the first processconfiguration; generating inspection data on the exterior of the targetobject from the one or more sets of images that have been processedunder the first process configuration; generating an inspection score bycomparing the inspection data on the exterior of the target object withfeature data, the feature data being pre-stored in a memory andincluding one or more predetermined exterior features of the targetobject; comparing the inspection score with a predetermined thresholdscore; setting the first process configuration as an optimalconfiguration if the inspection score satisfies the predeterminedthreshold score; and determining a second process configuration for theoperation if the inspection score does not satisfy the predeterminedthreshold score.

Specifically, the determining the first process configuration includes:determining a first set of images from the one or more sets of images,on which the operation under the first process configuration isperformed; determining a first set of image processes from the pluralityof image processes, which is performed in the operation under the firstprocess configuration; determining first parameter values for the firstset of image processes; and determining a first sequence of the firstset of image processes.

Specifically, the generating the inspection score includes: obtainingone or more flags from the inspection data, each of the one or moreflags indicating whether the exterior of the target object includes atleast one of the predetermined exterior features; determining whethereach of the one or more flags is consistent with the feature datapre-stored in the memory; and generating the inspection score thatindicates a ratio of a number of flags that are consistent with thefeature data to a total number of the one or more flags.

Specifically, the generating the inspection score includes: obtaining afirst area on the target object from the inspection data, the first areaincluding at least one of the predetermined exterior features; obtaininga second area on the target object from the feature data pre-stored inthe memory, the second area including the at least one of thepredetermined exterior features; determining an overlapping area betweenthe first area and the second area; and generating the inspection scorethat indicates a proportion of the overlapping area in the second area.

Specifically, the determining the second process configuration for theoperation includes determining a second set of images from the one ormore sets of images, a second set of image processes from the pluralityof image processes, second parameter values for the second set of imageprocesses, and a second sequence of the second set of image processes,according to a difference between the inspection score and thepre-determined threshold score.

Specifically, the determining the second process configuration for theoperation includes: deriving a difference between a previous inspectionscore, which has been generated using a previous process configurationfor the operation before performing the operation under the firstprocess configuration, and the inspection score generated using thefirst process configuration for the operation; and determining thesecond process configuration by modifying the first processconfiguration according to the derived difference.

Specifically, a method for optimizing an inspection of an exterior of atarget object further comprises: performing the operation under thesecond process configuration.

Specifically, the one or more predetermined settings include at leastone of a wavelength of light irradiated to the target object forcapturing the one or more sets of images of the target object, anirradiation angle of the light into the target object, an illuminationof the light, and a type of at least one camera configured to capturethe one or more sets of images of the target object.

Specifically, the first sequence includes a sub-sequence that applies atleast two different image processes, among the first set of imageprocesses, to the first set of images in parallel.

Specifically, the first sequence includes a sub-sequence that applies asingle image process, among the first set of image processes, to thefirst set of images after applying the at least two different imageprocesses to the first set of images in parallel.

Advantageous Effects of Invention

According to various embodiments of the present disclosure, anelectronic device can sense a defect of a target object accurately andefficiently by deriving an optimized configuration of an operation forinspecting a defect of the exterior of the target object.

According to various embodiments of the present disclosure, anelectronic device can accurately derive an optimal configuration, not aconfiguration corresponding to a local maxima or local minima, inconnection with optimization of an operation for inspecting a defect ofa target object.

According to various embodiments of the present disclosure, anelectronic device can derive an optimal configuration of a defectinspection operation according to an objective standard.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates an overall operating process of an electronic deviceaccording to various embodiments of the present disclosure.

FIG. 2 illustrates a block diagram of an electronic device according tovarious embodiments of the present disclosure.

FIG. 3 illustrates an operation for processing an image according to anembodiment of the present disclosure.

FIG. 4 illustrates a process that optimizes an operation for processingan image according to an embodiment of the present disclosure.

FIG. 5 illustrates a method for evaluating inspection data by aprocessor according to an embodiment of the present disclosure.

FIG. 6 illustrates a method for determining a parameter value by aprocessor during a process of determining a new configuration of anoperation according to an embodiment of the present disclosure.

FIG. 7 illustrates a branching sub-sequence and a merging sub-sequenceaccording to an embedment of the present disclosure.

FIG. 8 illustrates an embodiment of an operation optimization methodthat may be performed by an electronic device according to the presentdisclosure.

BEST MODE FOR CARRYING OUT THE INVENTION

Various embodiments disclosed in this document are illustrated for thepurpose of clearly describing the technical idea of the presentdisclosure, and are not to be limited to a specific mode of embodiment.The technical idea of the present disclosure includes variousmodifications, equivalents, and alternatives of each embodiment, and acombination of embodiments selected from all or part of each embodiment.In addition, the scope of the technical idea of the present disclosureis not limited to the following various embodiments or specificdescriptions thereof.

Terminology used in this document, including technical or scientificterms, may have meanings that are normally understood by a personskilled in the art to which the present disclosure pertains unlessotherwise defined.

Expressions such as “include”, “may include”, “comprise”, “maycomprise”, “have”, and “may have” used in this document mean that afeature in question (for example, function, operation, or element)exists, and do not exclude the existence of another feature. That is,such expressions are to be understood as open-ended terms implying thepossibility that another embodiment may be included.

Singular expressions used in this document may include plural meaningsunless otherwise indicated in the context, and this applies to singularexpressions used in the claims as well.

Expressions “first”, “second”, and the like in this document are used,in connection with denoting a plurality of entities of the same kind, todistinguish an entity from another entity, unless otherwise indicated inthe context, and do not limit the order or importance between theentities in question.

Expressions “A, B, and C”, “A, B, or C”, “A, B, and/or C”, “at least oneof A, B, and C”, “at least one of A, B, or C”, “at least one of A, B,and/or C” and the like used in this document may denote respectiveenumerated items or all possible combinations of the enumerated items.For example, “at least one of A or B” may denote all of the following:(1) at least one A, (2) at least one B, and (3) at least one A and atleast one B.

The expression “on the basis of” is used in this document to describeone or more factors that affect an action or operation of determinationor decision described in the phrase or sentence including theexpression, and this expression does not exclude additional factors thataffect the action or operation of determination fir decision inquestion.

The expression that an element (for example, first element) is“connected” to a different element (for example, second element) used inthis document may mean that the element is not only directly connectedto the different element, but also connected via another differentelement (for example, third element).

The expression “configured to” used in this document may have, dependingon the context, the meanings “set to”, “able to”, “modified to”,“adapted to”, “capable of”, and the like. This expression is not limitedto the meaning of “specially designed on a hardware basis”, and aprocessor configured to perform a specific operation, for example, maydenote a generic-purpose processor capable of performing the specificoperation by executing software.

Hereinafter, various embodiments of the present disclosure will bedescribed with reference to the accompanying drawings. In theaccompanying drawings and descriptions thereof, the same orsubstantially equivalent elements may be given the same referencenumerals. In the following description of various embodiments,furthermore, repeated descriptions of the same or corresponding elementsmay be omitted without meaning that the elements in question are notincluded in the corresponding embodiment.

FIG. 1 illustrates an overall operating process of an electronic deviceaccording to various embodiments of the present disclosure. Anelectronic device according to various embodiments of the presentdisclosure may perform optimization of an operation for processing aplurality of sets of images in order to inspect, a defect of theexterior of a target object. In some embodiments, the electronic deviceaccording to the present disclosure may inspect a defect of at least onetarget object that is given as a sample. Each of the at least one targetobject may or may not have a defect. Together with the at least onetarget object given as a sample, feature data 400 may be provided so asto confirm whether each target object actually has a defect and toindicate the actual location of the defect with regard to thecorresponding target object. The feature data 400 may be genuineinformation regarding the actual defect of the corresponding targetobject and/or information to be compared with a result value obtainedthrough a measurement. The electronic device according to the presentdisclosure may store the provided feature data.

An electronic device according to the present disclosure may obtain oneor more sets of images 100. The electronic device may shed light to atarget object 2 using a light source 30 (described later), and maycapture reflected light using a camera 40 (described later). Theelectronic device may obtain one or more sets of images 100 regardingthe target object using the captured light.

Thereafter, the electronic device according to the present disclosuremay start an operation 200 for inspecting a defect of the target object.This operation may be an operation based on a certain configuration. Theelectronic device may select one or more sets of images from a pluralityof sets of images according to the certain configuration, and may applypredetermined image processes to the selected sets of images in apredetermined sequence. The predetermined image processes may haveparameter values based on a definition included in the certainconfiguration.

The electronic device according to the present disclosure may obtaininspection data 300 according to an operation based on a certainconfiguration. The inspection data 300 may be measurement data resultingfrom an inspection of a defect of the corresponding target object by theelectronic device. The electronic device may compare the inspection data300 with stored feature data 400 and may modify the configuration of theoperation according to the result of comparison.

In one embodiment, the electronic device may automatically repeatapplication of the operation according to a certain configuration,evaluation of the operation according to the result of the correspondingoperation, and modification of the operation configuration based on theevaluation result. If an appropriate configuration of the operation isderived during the repeated processes, the corresponding configurationmay be determined as the optimized configuration of this operation, inthis manner, the electronic device according to the present disclosuremay optimize the operation for inspecting a defect of the target object.After the optimized configuration of the operation is determined, theelectronic device may proceed to inspect defects of other target objectsthrough an operation based on the optimized configuration.

According to various embodiments of the present disclosure, anelectronic device may sense a defect of a target object accurately andefficiently by deriving an optimized configuration of an operation forinspecting a defect of the target object.

According to various embodiments of the present disclosure, anelectronic device can accurately derive an optimal configuration, not aconfiguration corresponding to a local maxima or local minima, inconnection with optimization of an operation for inspecting a defect ofa target object.

According to various embodiments of the present disclosure, anelectronic device can derive an optimal configuration of a defectinspection operation according to an objective standard.

FIG. 2 illustrates a block diagram of an electronic device 1 accordingto various embodiments of the present disclosure. The above-describedelectronic device according to the present disclosure may be representedby the illustrated electronic device 1.

According to an embodiment, the electronic device 1 may include a lightsource 30, a camera 40, a memory 20, and/or a processor 10. In someembodiments, at least one of the above elements of the electronic device1 may be omitted, or another element may be added to the electronicdevice 1. In some embodiments, in addition or in alternative to thesame, some elements may be integrated and implemented, or may beimplemented as a single entity or a plurality of entities.

At least some elements among the elements inside/outside the electronicdevice 1 may be interconnected, for example, through a GPIO (generalpurpose input/output), a SPI (serial peripheral interface), and/or aMIPI (mobile industry processor interface) so as to exchange data and/orsignals.

The light source 30 may irradiate light to the target object. Theelectronic device 1 may include at least one light source 30, and eachof the at least one light source 30 may be configured variously. In anembodiment, the light source 30 may be omitted.

The camera 40 may capture one or more sets of images of the targetobject. The camera 40 may capture light irradiated by the light source30 and then reflected from the target object 2, thereby obtaining animage of the target object 2. This image may be included in the one ormore sets of images. The electronic device 1 may include at least onecamera 40, and each of the at least one camera 40 may be configuredvariously.

The memory 20 may store various types of information necessary to drivethe electronic device 1. Data stored in the memory 20 may data obtained,processed, or used by at least one element of the electronic device 1,and may include software (for example, programs). The memory 20 mayinclude a volatile memory and/or a non-volatile memory. The memory 20may store one or more sets of images of the target object. In addition,the memory 20 may store feature data regarding a defect part of thetarget object. The feature data may include information regarding one ormore predetermined exterior features or exterior defects of the targetobject.

The processor 10 may drive software (for example, programs) so as tocontrol at least one element of the electronic device 1 connected to theprocessor 10. In addition, the processor 10 may perform one or morevarious operations related to the present disclosure, such ascalculation, processing, data generation, and modification. In addition,the processor 10 may load data from the memory 20 or store the same inthe memory 20.

The processor 10 may obtain one or more sets of images 100 from thememory 20. Each of the sets of images may be one captured under one ormore predetermined settings.

The processor 10 may determine a first configuration of an operation forprocessing one or more sets of images 100. The processor 10 may processone or more sets of images according to an operation based on the firstconfiguration. The operation may include one or more image processes.The processor 10 may generate inspection data 300 regarding a defect ofthe target object from one or inure sets of images processed by theoperation based on the first configuration.

The processor 10 may generate an inspection score by comparinginspection data 300 with feature data 400 obtained from the memory 20.The inspection score may be generated by evaluating the degree ofefficiency to which the operation senses a defect of the target objectaccording to the first configuration, and then providing a numericalvalue that represents the result of evaluation.

The processor 10 may compare the inspection score with a predeterminedthreshold score. Depending on the comparison result, the processor 10may determine the first configuration as the optimal configuration ormay newly determine a second configuration. When the inspection scorederived according to the first configuration satisfies the predeterminedthreshold score, the processor 10 may determine the first configurationas the optimal configuration. In this case, the processor 10 may storethe first configuration in the memory according to an embodiment. Inaddition, when the inspection score derived according to the firstconfiguration fails to satisfy the predetermined threshold score, theprocessor 10 may determine a second configuration of an operation forprocessing one or more sets of images. The operation based on the secondconfiguration may be performed after the operation based on the firstconfiguration. The processor 10 may keep performing the operation withregard to the one or more sets of images while changing theconfiguration of the operation until the optimal configuration isderived. In an embodiment, the fact that the inspection score satisfiesthe threshold score may mean that the value of the inspection score ishigher than the threshold score. In an embodiment, the fact that theinspection score satisfies the threshold score may mean that the valueof the inspection score is lower than the threshold score. In anembodiment, the fact that the inspection score satisfies the thresholdscore may mean that the value of the inspection score exists within apredetermined percentage range of the predetermined threshold score.

In the present disclosure, the target object 2 may be various objectsthat are subject to a MOI. The target object 2 may, for example, be asemiconductor, a semiconductor substrate, and/or a die mounted on thesemiconductor substrate. The target object in the present disclosure isnot limited to the above-mentioned objects.

In the present disclosure, a defect may refer to a characteristic partof a target object, which can be found from the surface or interior ofthe target object. In an embodiment, the defect may refer to adifference in the surface or interior a target object compared with aspecification of the target object. In an embodiment, the defect may bea stain in a specific color appearing on the colorless surface of thetarget object. In an embodiment, the defect may be a scratch on thesurface of the target object. In an embodiment, the defect may be a dentor a bulge on the target object. In an embodiment, the defect may be afeature intentionally generated on the target object according to amachining process. In the present disclosure, the defect may also bereferred to as, for example, a feature, an anomaly, and/or animperfection.

In the present disclosure, an image of the target object 2 may beobtained by taking an image of the target object 2. Light irradiatedtoward the target object by the light source 30 may be reflected fromthe target object 2, and an image generated by capturing the reflectedlight by the camera 40 may be the image of the target object 2. In anembodiment, the image may be obtained by taking an image of a partialarea or ROI (Region Of interest) of the target object 2. There may be animage for each view based on each FOV (Field Of View) of the targetobject 2.

In the present disclosure, a set of images may be a set of images of thetarget object taken under a predetermined setting. For example, imagesof the target object taken under a certain setting may be included in asingle set of images. As used herein, the predetermined setting mayrefer to a set of values configured with regard to all factors relatedto taking images of the target object. The factors may include, forexample, the type, amount, and arrangement of the light source 30, thetype, amount, and arrangement of the camera 40, parameters related tothe light source 30, parameters related to the camera 40, the wavelengthof light, the illumination of light, the angle at which light isirradiated to the target object and any combinations thereof.

In the present disclosure, an operation may refer to a series ofprocesses for processing a set of images of the target object andsensing a defect of the target object from the set of images, in orderto inspect a defect of the target object. The operation may be for theprocessor to apply at least one set of image processes to predeterminedsets of images, among one or more sets of images, according to apredetermined sequence. The content of the operation may vary dependingon how the operation is configured. As described above, theconfiguration of the operation may include configurations regarding, forexample, how to determine a set of images to which the operation is tobe applied; how to determine a set of image processes included in theoperation, among a plurality of image processes; how to determine valuesof parameters used for the set of image processes; and in what sequencethe image processes of the set of image processes are to be applied. Inthe present disclosure, the configuration may also be referred to as aprocess configuration.

In the present disclosure, an image process may be a conceptencompassing all image processing applied to images of a target objectin order to inspect a defect of the target object. The concept of theimage process may include, for example, data processing, an imageprocessing algorithm, and/or a data processing algorithm. At least oneimage process may belong to the operation. The type of image processeswill be described later.

In the present disclosure, inspection data may refer to a result ofmeasurement regarding a defect of a target object, inspected by applyingan operation to an image of the target object. The inspection data maybe a result obtained by processing one or more sets of images accordingto an operation based on a certain configuration by the processor 10.The inspection data may include a flag. The flag of the inspection datamay indicate whether a defect exists on the corresponding target object.In an embodiment, the flag may be a binary indicator. In an embodiment,the inspection data may include a feature location image. The featurelocation image of the inspection data may indicate an area of a targetobject that is occupied by a defect existing on the corresponding targetobject. In an embodiment, the feature location image may be representedin a heat-map image type, which is similar to the above-mentioned targetobject image. In an embodiment, the feature location image may bereferred to as a spatial location map. The inspection data is obtainedby applying an operation to one or more sets of images, and thus mayinclude a plurality of flags and/or a plurality of feature locationimages, depending on continuous image inputs.

In the present disclosure, the feature data 400 may include informationregarding an actual defect of the target object. The feature data 400may be provided together with a target object, which is given as asample, and may include genuine information regarding a defect existingon the target object. The feature data may include information regardingwhether the corresponding target object has a defect and informationregarding the location of the defect on the target object. The locationinformation may indicate the area on the surface of the target object inwhich the defect exists. In an embodiment, the feature data 400 may beinformation generated by a human after visually inspecting the targetobject 2 or information generated by equipment such as a computer.

In the present disclosure, an inspection score may be data indicating,by a numerical value, how accurately inspection data has sensed a defectof the target object, the inspection data being derived when anoperation based on a certain configuration is performed with regard toone or more sets of images. The inspection score may be obtained bycomparing inspection data derived according to an operation based on acertain configuration and feature data that was stored in advance. Whenthe inspection data derived according to an operation based on a certainconfiguration is a flag, the inspection score may be represented by aratio of the number of flags that are consistent with the content of thefeature data to the number of one or more flags of the inspection data.When the inspection data derived according to an operation based on acertain configuration is a feature location image, in an embodiment, theoperation may first determine an overlapping area between an area inwhich a defect indicated by the feature location image of the inspectiondata exists and an area in which a defect indicated by the feature dataexists. The inspection score may then be represented by a proportion ofthe overlapping area to the area in which the defect indicated by thefeature data exists. This proportion, numerical value data, may be usedto determine the accuracy of the operation based on the certainconfiguration in question. The inspection score may also be referred toas, for example, evaluation data and/or a score.

In this disclosure, a predetermined threshold score, which is comparedwith the inspection score, may be a reference value for determining acertain configuration of an operation as the optimal configuration. Asdescribed above, when an inspection score obtained by an operation basedon a certain configuration satisfies the predetermined threshold score,the corresponding certain configuration may be determined as the optimalconfiguration. The predetermined threshold score may be appropriatelydetermined according to a desired degree of optimization of theoperation.

In the present disclosure, a program refers to software stored in thememory 20, and may include an operating system for controlling resourcesof the electronic device 1, applications, and/or middleware thatprovides the applications with various functions such that theapplications can utilize the resources of the electronic device 1.

In the present disclosure, the electronic device 1 may be various typesof devices. For example, the electronic device may be a portablecommunication device, a computer device, a portable multimedia device, awearable device, or a device based on a combination of one or more ofthe above-mentioned devices. The electronic device according to thepresent disclosure is not limited to the above-mentioned devices. Theelectronic device 1 may also be simply referred to as a device.

In an embodiment, the electronic device 1 may include a plurality oflight sources 30. Each of the plurality of light sources 30 may beconfigured differently. In an embodiment, the light sources 30 may betwo-dimensional light sources that irradiate RGB (Red, Green, Blue)light or three-dimensional light sources that irradiate pattern light.In an embodiment, each of the plurality of light sources 30 may bepositioned on a normal line of the target object 2 so as to irradiatelight perpendicularly on the target object 2, or may be positioned at apoint inclined by a predetermined angle from the normal line of thetarget object 2 so as to obliquely irradiate light on the target object2. The angle of light irradiated to the target object 2 may differdepending on the arrangement of each of the plurality of light sources30. In an embodiment, light irradiated by each of the plurality of lightsources 30 may have a different wavelength. In an embodiment, lightirradiated by each of the plurality of light sources 30 may have adifferent illumination.

In an embodiment, the electronic device 1 may include a plurality ofcameras 40. Each of the plurality of cameras 40 may be configureddifferently. In an embodiment, each of the plurality of cameras 40 mayhave a different type. For example, one or inure of the cameras 40 maybe an image sensor such as a CCD (Charged Coupled Device) and a CMOS(Complementary Metal-Oxide-Semiconductor). In an embodiment, each of theplurality of cameras 40 may be arranged in various types near the targetobject to be able to capture light reflected from the target object.

In an embodiment, the memory 20 may store a plurality of image processesthat may be applied to one or more sets of images.

In an embodiment, the processor 10 may determine, in order to determinea first configuration of an operation, a first set of images to whichthe operation based on the first configuration is to be applied, fromone or more sets of images.

In an embodiment, the processor 10 may determine, in order to determinea first configuration of an operation, a first set of images included inthe operation based on the first configuration, from a plurality ofimage processes.

In an embodiment, the processor 10 may determine, in order to determinea first configuration of an operation, first parameter values of one ormore parameters of image processes included in the determined first setof image processes.

In an embodiment, the processor 10 may determine, in order to determinea first configuration of an operation, a first sequence in which imageprocesses included in the determined first set of image processes areapplied to the first set of images during the operation.

In an embodiment, the processor 10 may process a plurality of sets ofimages according to the first configuration of the operation.Specifically, the processor 10 may apply a first set of image processes,which have first parameter values as parameters, according to the firstsequence, thereby processing the first set of images.

In an embodiment, the processor 10 may obtain one or more flags from theinspection data in order to compare the feature data with the inspectiondata and to generate an inspection score. Each of the one or more flagsmay indicate whether the exterior of the target object has apredetermined exterior feature or exterior defect. The processor 10 maydetermine whether each of the one or more flags is consistent withinformation regarding a defect, which is indicated by feature datapre-stored in the memory. To this end, the processor 10 may compare thefeature data with each of the one or more flags. The processor 10 maygenerate an inspection store that indicates the ratio of the number offlags that are consistent with the feature data to the number of flagsinside the inspection data.

In an embodiment, the processor 10 may obtain a feature location imagefrom the inspection data, in order to compare the feature data with theinspection data and to generate an inspection score. The featurelocation image may have information regarding a first area, and thefirst area may include at least one predetermined exterior feature orexterior defect on the exterior of the target object. In addition, theprocessor 10 may obtain a second area from feature data pre-stored inthe memory. The second area may include at least one predeterminedexterior feature or exterior defect. That is, the first area may be adefect area appearing as a result of inspection or measurement accordingto the operation, and the second area may be an actual defect area of atarget object sample pre-stored in the memory. The processor 10 mayderive an overlapping area between the first area and the second area.The processor 10 may generate an inspection score that indicates theproportion of the overlapping area to the second area in which a defectindicated by the feature data exists.

In an embodiment, the processor 10 may determine the secondconfiguration of the operation by comparing the inspection score, whichis derived according to the first configuration of the operation, withthe above-mentioned predetermined threshold score. In an embodiment, theprocessor 10 may determine the second configuration of the operation onthe basis of a difference between the inspection score based on thefirst configuration and the above-mentioned predetermined thresholdscore. Specifically, on the basis of the above-mentioned difference, theprocessor 10 may determine a second set of images to which the operationbased on the second configuration is applied, from a plurality of setsof images; the processor 10 may determine a second set of imageprocesses included in the operation based on the second configuration,from a plurality of image processes; the processor 10 may determinesecond parameter values of parameters of the second set of imageprocesses; and the processor 10 may determine a second sequence in whichthe second set of image processes are applied. As used herein, thesecond configuration may refer to a configuration applied to anoperation for processing a plurality of sets of images, following theoperation based on the first configuration.

In an embodiment, in connection with determining the secondconfiguration of the operation, the processor 10 may modify the firstconfiguration based on how the inspection score changes as theconfiguration of the operation is changed, thereby determining thesecond configuration. The processor 10 may initially apply an operationbased on a certain configuration to a plurality of sets of images,thereby deriving an inspection score. Following the operation based onthe certain configuration, the processor 10 may apply the operationbased on the first configuration to the plurality of sets of images.

The processor 10 may derive an inspection score based on the firstconfiguration, and may compare the same with an inspection score basedon a certain configuration applied prior to the first configuration. Theprocessor 10 may measure how the inspection score changes in thiscomparison process and may modify the first configuration on the basisof the change of the inspection score, thereby determining the secondconfiguration. The change of the inspection score may include adifference between the two inspection scores.

In an embodiment, in connection with determining the secondconfiguration of the operation, the processor 10 may determine detailsof the second configuration in various methods on the basis of the firstconfiguration used prior to the second configuration. For example, theprocessor 10 may determine a set of images, a set of image processes,parameter values, and/or a sequence of the set of image processes thatare used for the second configuration, in various methods. To this end,methods such as trial and error manner, gradient ascent/descent,annealing, and/or Gibbs measure may be used. These methods will bedescribed later.

In an embodiment, the processor 10 may perform the operation based onthe second configuration, which is determined based on the inspectionscore of the first configuration. That is, the processor 10 may apply asecond set of image processes, which have a second parameter value as aparameter, according to the second sequence, thereby processing thesecond set of images.

In an embodiment, the processor 10 may repeatedly perform the task ofdetermining a configuration to be used for the next operation based onthe inspection score of the previous configuration. This repeated taskmay continue until a configuration is determined as the optimalconfiguration according to the above-mentioned process.

In an embodiment, a sequence of an operation based on a certainconfiguration may include at least one sub-sequence. A predetermined setof images during an operation may undergo a plurality of imageprocesses. In this case, the predetermined set of images may undergoeach of the plurality of image processes according to the sequence. Inan embodiment, a set of images may undergo the plurality of imageprocesses in a parallel manner during an intermediate step of theoperation; the portion of the sequence that branches as described abovemay be referred to as a branching sub-sequence. In an embodiment, a setof images that have undergone the image processes in a parallel mannermay undergo a single image process; the portion of the sequence thatmerges as described above may be referred to as a merging sub-sequence.

In an embodiment, the electronic device 1 may further include acommunication interface (not illustrated in FIG. 2). The communicationinterface may perform wired or wireless communication between theelectronic device 1 and the server or between the electronic device 1and a different external electronic device. For example, thecommunication interface may perform wireless communication according toa scheme such as LTE (long-term evolution), LTE-A (LTE Advance), CDMA(code division multiple access), WCDMA (wideband CDMA), WiBro (WirelessBroadband), WiFi (wireless fidelity), Bluetooth, NFC (near fieldcommunication), GPS (Global Positioning System), or GNSS (globalnavigation satellite system). For example, the communication interfacemay perform wired communication according to a scheme such as USB(universal serial bus), HDMI (high definition multimedia interface),RS-232 (recommended standard 232), or POTS (plain old telephoneservice).

In an embodiment, the processor 10 may control the communicationinterface so as to obtain information from the server. The informationobtained from the server may be stored in the memory 20. In anembodiment, the information obtained from the server may include, forexample, the above-mentioned feature data, the above-mentioned pluralityof image processes, and/or the above-mentioned one or more sets ofimages.

In an embodiment, the above-mentioned operation optimization process maybe performed in a parallel manner by a plurality of electronic devices.Specifically, the process of evaluating the configuration of theoperation and determining the next configuration of the operation, bythe processor 10, may be performed in a parallel manner by a pluralityof electronic devices, in order to utilize the computing power of theplurality of electronic devices. This may enable the process of derivingthe optimal configuration of the operation to proceed in a rapid manner.The result of the optimization process performed in a parallel mannermay be aggregated and processed by a single electronic device, and maybe used to derive the optimal configuration regarding the operation. Inan embodiment, the operation optimization process performed in aparallel manner may be performed by a plurality of electronic devicesinstalled remotely. That is, the operation optimization processperformed in a parallel manner may be performed by cloud computing.

In an embodiment, the above-mentioned operation optimization process maybe performed by the processor 10 inside the electronic device 1, inorder to improve or guarantee data security.

In an embodiment, the processor 10 may control the light source 30and/or the camera 40 on the basis of the inspection data and/or theinspection score.

In an embodiment, when the processor 10 has repeated the task fordetermining the optimal configuration of the operation by apredetermined number of times, but has failed to derive a configurationdeemed to be the optimal configuration, the processor 10 may control thelight source 30 and/or the camera 40 so as to obtain an additional setof images.

In an embodiment, when the processor 10 needs predetermined linage datain the process of determining the detail of the next configuration ofthe operation, the processor 10 may control the light source 30 and/orthe camera 40 so as to obtain the necessary image.

In an embodiment, the processor 10 may control the light source 30and/or the camera 40 so as to change, for example, the arrangement ofthe light source 30, the arrangement of the camera 40, parametersrelated to the light source 30, parameters related to the camera 40, thewavelength of light, the illumination of light, and/or the angle atwhich light is irradiated to the target object.

In an embodiment, the electronic device 1 may further include an inputdevice (not illustrated in FIG. 2). The input device may be a device forreceiving data to be transferred to at least one element of theelectronic device 1 from the outside. For example, the input device mayinclude a mouse, a keyboard, and/or a touch pad.

In an embodiment, the electronic device 1 may further include an outputdevice (not illustrated in FIG. 2). The output device may be a devicefor providing the user with various types of data, such as the result ofinspection by the electronic device 1 and the operating conditionthereof, in a visual type. For example, the output device may include adisplay, a projector, and/or a hologram.

In an embodiment, a plurality of different operations may be used. Itmay be necessary to apply different operations, in order to sense adefect from taken images of respective parts of the target object,according to features of respective parts of the target object. In thiscase, operations applied to images of respective parts of the targetobject may need to have different configurations. Accordingly, theelectronic device 1 may perform an optimization process independently,according to images of respective pails of the target object. In anembodiment, a separate operation or a separate configuration may beapplied with regard to each ROI of the target object. The electronicdevice 1 may accordingly perform an optimization process independentlywith regard to each ROI. In an embodiment, an operation having theoptimal performance may be selected from operations that have undergonerespective optimization processes.

In an embodiment, the electronic device 1 may be implemented by adigital computer system.

In an embodiment, the processor 10 may derive the optimal configurationaccording to a gradient descent method (described later).

In an embodiment, the processor 10 may derive the optimal configurationaccording to a simulated annealing method (described later).

In an embodiment, at least part of the calculation performed by theprocessor 10 may be performed by a GPU (Graphical Processing Unit). Thatis, the processor 10 may be at least partially or entirely implementedby a GPU.

Various embodiments of the electronic device 1 according to the presentdisclosure may be combined with one another. Respective embodiments maybe combined in any permutation, and any embodiment of the electronicdevice 1 made by such a combination also belongs to the scope of thepresent disclosure. In addition, the internal/external elements of theelectronic device 1 according to the present disclosure disclosed above,may be added, modified, respaced, or deleted according to theembodiment. In addition, the internal/external elements of theelectronic device 1 disclosed above, may be implemented as hardwarecomponents.

FIG. 3 illustrates an operation for processing an image according to anembodiment of the present disclosure. As described above, an operationmay refer to a process of processing a set of images of a target object,in order to inspect a defect of the target object, and obtainingmeasurement data (that is, inspection data) regarding the defect. In theillustrated embodiment, the operation according to the presentdisclosure may refer to an operation based on a predeterminedconfiguration. According to this configuration, the operation mayinclude predetermined image processes 210, 220, and 230 to be applied toa plurality of sets of images 100.

Initially, the processor 10 may obtain a plurality of sets of images100. The processor 10 may successively apply an image process of anoperation based on a predetermined configuration to the plurality ofsets of images. In this predetermined configuration, the operation mayinclude three image processes 210, 220, and 230. In an embodiment, theimage process 210 may correspond to a selection process for selecting apredetermined set of images to which the operation based on thecorresponding configuration is to be applied, among the plurality ofsets of images 100. The selection process may select a predetermined setof images 110 from the plurality of sets of images 100 that have beeninput, according to the corresponding configuration of the operation,and may output the selected set of images.

The processor 10 may successively apply the next image processes 220 and230 to the predetermined set of images 110 that have been output,thereby processing the set of images. In an embodiment, as illustrated,the number of sets of images may be reduced as the sets of images areprocessed according to respective image processes. Image processes usedin this operation may be those selected from a plurality of imageprocesses according to the above-mentioned predetermined configuration.In addition, the sequence of application of the image processes used inthis operation may be determined according to the above-mentionedpredetermined configuration.

The processor 10 may extract inspection data 300 from the sets of imagesthat have undergone respective image processes of the operation. Theinspection data 300 may be information regarding a defect of the targetobject obtained from sets of images of the target object, as mentionedabove.

In the illustrated embodiment, the processor 10 may not evaluate whetherthe inspection data is accurately describing the defect of the targetobject, or may not modify the configuration of the illustrated operationaccording to the result of evaluation. In such an embodiment, forexample, a person may compare the inspection data with the feature dataso as to evaluate the accuracy of the inspection data.

FIG. 4 illustrates a process that optimizes an operation for processingan image according to an embodiment of the present disclosure. Asdescribed above, the electronic device 1 or processor 10 according tothe present disclosure may optimize an operation for inspecting a defectof a target object. For the purpose of the optimization process, theprocessor 10 may perform a task of finding the optimal configurationwhile changing the configuration of the operation for processing a setof images of the target object. The illustrated embodiment hasadditional operations 250, 260, 270, and 280 for optimizing theconfiguration of the operation, compared with the above-describedembodiment.

The processor 10 may evaluate 250 inspection data 300 resulting from theoperation based on the current configuration. The processor 10 maycompare the inspection data 300 with feature data 400 in this evaluationprocess 250. According to the result of comparison, the processor 10 maygenerate an inspection score 500 that indicates the accuracy of theinspection data 300.

The inspection score 500 may be a numerical representation of howaccurately the operation based on the current configuration senses adefect of the target object. As mentioned above, when the inspectiondata 300 is a flag that indicates whether the target object has adefect, the processor 10 may determine, as the inspection score, theratio of the number of flags that identically indicate whether thetarget object has a defect as the feature data, to the number of theflags of the inspection data. As described above, when the inspectiondata 300 is a feature location image, the processor 10 may determine, asthe inspection score, the ratio of an overlapping area in which a defectindicated by the feature data exists to an area of the feature locationimage.

The processor 10 may compare the inspection score 500 with apredetermined threshold score that has been defined in advance and maydetermine, according to the result of comparison, whether to store thecurrent configuration in the memory 20 or to determine 260 a newconfiguration. The inspection data evaluation process 250 by theprocessor 10 may be performed by an evaluator module. As used herein, a“module” may refer to a logic, a logical block, a component, or acircuit implemented as software or firmware. The module may constitutean integrally configured component, a smallest unit that performs one ormore functions, or a part thereof. For example, the module may beconfigured by an ASIC (application-specific integrated circuit). In anembodiment, the operation of the evaluator module may be performed bythe above-mentioned processor 10.

The processor 10 may determine 270 a configuration (for example, secondconfiguration) of an operation to be applied after the currentconfiguration (for example, first configuration) on the basis of variousdata. For example, the processor 10 may determine the next configurationof the operation in view of the inspection score based on the currentconfiguration and/or the tendency of change of the inspection scorebased on configurations.

As described above, determining a configuration of an operation by theprocessor 10 may include the following: determining (selecting) a set ofimages (for example, second set of images), to which the operation basedon the configuration in question is to be applied, from a plurality ofsets of images; determining (selecting) a set of image processes (forexample, second set of image processes), which are to be applied to theset of images, from a plurality of image processes: determining asequence (for example, second sequence) to be applied to the determinedset of image processes and deter a parameter value (for example, secondparameter value) to be used for the determined set of image processes.

The processor 10 may apply 280 the operation to a plurality of sets ofimages 100 according to the determined next configuration (for example,second configuration). That is, the processor 10 may apply a second setof image processes, which have a second parameter value as a parameter,to the second set of images according to the second sequence. Withreference to inspection data obtained from the operation based on thenext configuration (for example, second configuration), the processor 10may determine whether to store the next configuration (for example,second configuration) as the optimal configuration again or to determinea new configuration (for example, third configuration).

The process 270 of optimizing the operation configuration by theprocessor 10 may be performed by an optimizer module. Modules have beendescribed above. In an embodiment, the operation by the optimizer modulemay be performed by the above-mentioned processor 10.

In an embodiment, the processor 10 may also change 290 the manner ofcalculating inspection data 300 according to the operationconfiguration. For example, the processor 10 may assign a separateweight in connection with calculating the inspection score according tothe definition made by the operation configuration, may exclude specificdata when calculating the inspection score, or may employ a differentstandard when comparing feature data and inspection data.

In an embodiment, each image process may also be implemented by ahardware component such as FPGA (Field Programmable Gate Arrays). In anembodiment, each image process may also be performed by a separateindependent electronic device (for example, computer). In an embodiment,sets of images before/after processing by each image process may betransferred between separate electronic devices that perform each imageprocess in various types (for example, Ethernet or WiFi). In anembodiment, respective elements (for example, image process, evaluationprocess 250, and/or optimization process 270) of the operation may beimplemented by a remote server, and may communicate with each other by atype such as the Internet. An optimal configuration may be selected fromconfigurations derived according to the optimization process performedby each electronic device, and this optimal configuration may be used asthe initial value of the next optimization process.

In an embodiment, each image process may be various image processingand/or data selection processing. For example, each image process may beone of or any combination of the followings: data selection operators, amorphological operator, elementwise lookup tables, convolutionaloperators, histogram calculation and back-projection operators,highpass/lowpass/bandpass filters, threshold operators, noise reductionalgorithms, complex operators, pixel level classifiers, blob detection,labeling algorithms, contour detection, shape estimation algorithms,and/or pixelwise arithmetic operators. The image process according tothe present disclosure is not limited to the above-mentioned imageprocesses.

The data selection operator may constitute an image process forselecting predetermined images from a plurality of images. The dataselection operator may correspond to the above-mentioned selectionprocess. In an embodiment, the data selection operator may select apredetermined set of images from a plurality of sets of images. Assuminga container having a plurality of taken images of an area of a targetobject, the data selection operator may select a predetermined subset ofimages from the container and may return the selected subset to thecontainer. In an embodiment, this container may include a plurality ofimages of the target object obtained under various settings. Thiscontainer may correspond to the above-mentioned memory 20. In anembodiment, the data selection operator may include at least oneparameter. Each parameter may be used to determine which image is to beselected from inputted images. In an embodiment, this parameter may be abinary parameter. In an embodiment, the data selection operator mayapply additional processing to an inputted set of images, in order toreduce the outputted set of images. For example, the data selectionoperator may derive a weighted mean of the inputted set of images andmay output the derived mean value. For example, the data selectionoperator may have a set of floating point-type parameters, in order toindicate a weight necessary for combination of images on the basis ofthe weighted means.

The morphological operator may perform an operation for mainlyprocessing a binary image together with a structuring element using anoperator such as intersection, union, inclusion, or complement. As usedherein, the binary image may refer to an image expressed only by twodistinguishable pixel values. The morphological operator may constitutean image process that processes a target object appearing on the binaryimage on the basis of a feature of the shape of the target objectindicated by the structuring element. The morphological operator mayinclude, according to the embodiment, processing such as erosion thatreduces the image area, dilation that increases the image area, or amethod that uses a morphological gradient, which is a difference betweenthe erosion and dilation. The morphological operator may also be usedfor a grayscale image and may have as parameters, for example, theradius of the target object indicated by the structuring element of themorphological operator and/or the shape thereof.

The elementwise lockup tables image process may be image processing thatdiversifies the value of each pixel of an image according to adefinition made in a lookup table. The elementwise lookup tables imageprocess may change the contrast of the image. The parameter of theelementwise lookup tables image process may be a parameter of a functiondefined by the lookup table. Values of the lookup tables may beexpressed by a curve based on at least one curvature parameter.

The convolutional operators may constitute an image process that appliesa convolutional filter to an input image. The convolutional operatorsmay be, for example, blur operators, sharpening and/or band-passoperators. Parameters of the convolutional operators may include thekernels size (radius) of image convolution and/or the intensity of theused convolutional filter.

The histogram backprojection operators may constitute an image processthat changes an image by replacing pixels according to the ratio ofnumber of the pixels on a histogram. In this case, the histogram may becalculated with regard to pixels corresponding to the entire image or apartial area of the entire image. The histogram backprojection operatorsmay have, as parameters thereof, the size and shape of the image forwhich the histogram is calculated, the bin (interval) of the histogramand the like.

The highpass/lowpass/bandpass filters may constitute an image processthat filters an inputted image in a frequency area. Thehighpass/lowpass/bandpass filters may, in that order, filter thehigh-frequency area, the low-frequency area, and a specific frequencyarea of an image expressed in the frequency area. An image, the filteredfrequency area of which has disappeared, may be again converted to atime domain and outputted. The highpass/lowpass/bandpass filters mayhave, as parameters, frequency values that become the standard offiltering.

The threshold operators may constitute an image process that converts animage, which has been quantized at a large number of levels, to an imagequantized at two levels. That is, the threshold operators may assign anew pixel value on the basis of a comparison with a constant value,thereby obtaining the result value. This variable may be referred to asa threshold. This variable may be equally applied throughout the entireimage, or may adaptively change according to the relative feature ofeach pixel periphery of the image. The threshold operators may have thisvariable as a parameter. The value of this variable may be given;alternatively, the same may be computed dynamically in order to ideallydivide the image histogram as in the case of Otsu threshold algorithm.The threshold operators may further include, as parameters, the size andshape of the pixel periphery area, which are considered when calculatingan adaptively changing variable.

The noise reduction algorithms may constitute an image process forremoving small points mainly generated by thermal noise. In anembodiment, the noise reduction algorithms may constitute a medianfilter for removing speckles. The median filter may replace a givenpixel value with a median value of pixels in the periphery of thecorresponding pixel. The noise reduction algorithms may be applied to abinary image. Each point of the image may be analyzed individually, andmay be filtered according to the feature of each. The noise reductionalgorithms may have, as parameters, the size and shape of the pixelperiphery area for which the median value is calculated. In addition,the noise reduction algorithms may have, as parameters, the referencesize value of points or blobs to be removed.

The complex operators may constitute an image process implemented byartificial neural networks. The complex operators may keep optimizingtheir parameters by using a training based on the artificial neuralnetworks. Such a training may be performed inside the image process ormay be performed on the basis of an inputted external parameter. Forexample, the complex operators based on the artificial neural networksmay be trained such that the same are optimized to sense a specificpattern of an image. In this case, the complex operators based on theartificial neural networks may receive an inputted image and may output,for example, whether the specific pattern is sensed throughout theentire image or in a heat-map type.

The pixel level classifiers may constitute an image process that definespixel levels and classifies pixels of an input image with regard to eachpixel level. The pixel level classifiers may be utilized in a mannersimilar to that of the image process using artificial neural networks.For example, the pixel level classifiers may be tree classifiers thatdistinguish pixels according to a classification tree, and the treeclassifiers may be used to obtain an estimate regarding whether a givenarea of an image includes a defect. The pixel level classifiers mayoutput, as a heat-map, the evaluation value of the result of applyingthe corresponding image process to the entire image. The pixel levelclassifiers may have, as parameters, the accuracy, the sensitivity,and/or the size and type of a window that determines application targetpixels.

The blob detection and labeling algorithms may constitute an imageprocess that senses blobs from an inputted binary image and outputs animage that represents the blots together with continuous numbersindicate the corresponding blobs. In an embodiment, the inputted binaryimage may include a plurality of blobs that are not connected. Theoutputted image may be shaped such that respective blobs of the imageare indicated by distinguished integer pixel values. The blob detectionarid labeling algorithms may have, as parameters, features of blots tobe found. For example, the parameters may include the size, area, and/orconvexity of the blobs.

The contour detection and shape estimation algorithms may constitute animage process that analyzes and outputs a specific shape or contour, asin the case of the blob detection. This image process may analyze aspecific shape or contour of an inputted image and may delete allnon-convex blobs from the input image.

The pixelwise arithmetic operators may constitute an image process thatperforms an arithmetic operation with regard to each pixel of an inputimage and outputs the result value. A logical operator may be used forsuch an operation. In an embodiment, the pixelwise arithmetic operatorsmay be performed between different images. In an embodiment, thepixelwise arithmetic operators may process respective pixels usingseparate metadata. For example, the pixelwise arithmetic operators mayset all pixels, which are not included in a specific area defined by themetadata, to zero.

FIG. 5 illustrates a method for evaluating inspection data by aprocessor according to an embodiment of the present disclosure. Asdescribed above, the processor 10 may generate an inspection score bycomparing inspection data of an operation based on a predeterminedconfiguration with feature data. The processor 10 may use variousmethods in order to generate an inspection score by evaluatinginspection data.

In an embodiment, when inspection data includes a flag, the processor 10may generate an inspection score using the flag.

Initially, a taken image of a predetermined part of a target object maybe processed by an operation based on a certain configuration.Inspection data obtained as a result of the processing may include aflag that indicates whether the predetermined part includes a defect.

Assuming that the predetermined part has an actual defect, and that theflag of the inspection data indicates that there is no defect, in otherwords, when an operation based on a certain configuration fails todetect the defect, the processor 10 may classify the flag as FN (FalseNegative). Assuming that the predetermined part has an actual detect,and that the flag of the inspection data indicates that there is adefect, in other words, when an operation based on a certainconfiguration has successfully detected the defect, the processor 10 mayclassify the flag as TP (True Positive). Assuming that the predeterminedpart has no actual defect, and that the flag of the inspection dataindicates that there is a defect, in other words, when an operationbased on a certain configuration has erroneously detected a defect, theprocessor 10 may classify the flag as FP (False Positive). Assuming thatthe predetermined part has no actual defect, and that the flag of theinspection data indicates that there is no defect, in other words, whenan operation based on a certain configuration has successfully confirmedthat the operation has no defect, the processor 10 may classify the flagas TN (True Negative). In this regard, the processor 10 may confirmwhether the predetermined part of the target object has an actual defectwith reference to the feature data 400.

The processor 10 may calculate respective classified flags as: in thecase of illustrated equation 5010. In the illustrated equation, TP, TN,FP, and FN may denote the number of flags classified as TP, TN, FP, andFN, respectively. The result value S of the equation 5010 may be anembodiment of the above-mentioned inspection score. The result value Smay denote the ratio of flags that are consistent with the content ofthe feature data among one or more flags the inspection data has.

When the inspection data includes a feature location image, theprocessor 10 may generate an inspection score using the feature locationimage. A taken image of a pre-determined part of a target object may beprocessed by an operation based on a certain configuration, andinspection data including a feature location image may be obtained. Inan embodiment 5030 of the illustrated feature location image, the defectpart of the target object appears as a white line on the image. Inaddition, in an embodiment 5030 of illustrated feature data, a defectpart actually existing on the target object may appear as a white lineon the image. In an embodiment, defect parts indicated by the featurelocation image and the feature data may appear as predetermined areas,for examples, areas marked by white color.

The processor 10 may derive an overlapping area between the area inwhich a defect indicated by the feature location image 5030 appears andan area in which a defect indicated by the feature data 5020 appears.The processor 10 may generate an inspection score on the basis of theoverlapping area.

In an embodiment, the inspection score may be determined as a proportionof the overlapping area to the area in which the defect indicated by thefeature data exists.

In an embodiment, the processor 10 may use an intersection-over-unionmeasure in connection with determining the inspection score. The areahaving a defect indicated by the feature location image of theinspection data may be labeled H, and the area having a defect indicatedby the feature data may be labeled L. Each area may be expressed by apixel number. In this case, an inspection score derived by theintersection-over-union measure may be represented as in the illustratedequation 5040. This inspection score may have a value between 0.0 and1.0. In the illustrated equation, the numerator may denote the number ofpixels belonging to both H and L areas. That is, the numerator may beidentical to a result of applying pixelwise “and” operator to bothareas. In the illustrated equation, the denominator may denote thenumber of pixels belong to one of areas H and L. That is, thedenominator may be identical to a result of applying pixel-wise “or”operator to both areas.

In an embodiment, the intersection-over-union measure may be performedwith regard to each image of each part of the target object, and theresult value may be aggregated with regard to the entire part. In anembodiment, an image of the entire part of the target object may bederived, and the intersection-over-union measure may be performed withregard to the image of the entire part.

The method for evaluating inspection data and generating an inspectionscore by the processor 10 is not limited to the above-mentioned methods.

FIG. 6 illustrates a method for determining a parameter value by aprocessor during a process of determining a new configuration of anoperation according to an embodiment of the present disclosure. Asmentioned above, the processor 10 may determine a configuration to beapplied after the current configuration, in order to find an optimalconfiguration of an operation for inspecting a defect of a targetobject. In the process of determining the next configuration, theprocessor 10 may determine parameter values of parameters to be appliedto image processes of the operation based on the next configuration. Theprocessor 10 may find an optimal configuration that enables theoperation to inspect a defect of the target object most accurately, bydetermining appropriate parameter values. That is, the processor 10 mayfind parameter values that make the above-mentioned inspection datamaximum. To this end, the processor 10 may keep determining parametervalues using various methods.

Methods for determining parameter values may largely include discretemethods and continuous methods. When a discrete method is followed, theprocessor 10 may keep finding a parameter value that gives the optimalresult while continuously changing the parameter value by apredetermined gradient. When a partial subset among a plurality ofparameters corresponds to a discrete-type parameter, at least a part ofthe optimization process may have to undergo an optimization processbased on the discrete method. In the case of the discrete method, it maybe difficult to efficiently estimate or determine a gradient value thatis ideal for optimization. When a continuous method is followed, theprocessor 10 may keep finding a parameter that gives the optimal resultwhile continuously changing the parameter value. It may be possible tofind the optimal parameter value, regardless of whether the discretemethod or the continuous method is used.

The processor 10 may utilize trial and error manner in order to find anoptimal parameter. According to this trial and error method, theprocessor 10 may randomly determine a parameter value and may evaluateinspection data 300 derived by performing an operation using the value,thereby continuously finding the optimal parameter value. According tothe result of evaluating the inspection data 300, that is, theinspection score 500, the process or 10 may again perform the operationusing a new parameter value. The processor 10 may repeatedly perform thetrial and error method.

When a more accurate defect inspection result is derived as a result ofperforming an operation after configuring a certain parameter value withregard to a predetermined parameter, that value may be determined as theoptimal value of the corresponding parameter, and optimization may beperformed through the trial and error manner with regard to a differentparameter value. In this case, alternatively, the processor 10 mayanalyze a certain parameter value that enables derivation of a moreaccurate defect inspection result and a parameter value that has beenapplied prior to the same, thereby deriving the tendency, and maydetermine the next parameter value to be applied to the correspondingparameter on the basis of the tendency. As used herein, the tendency mayindicate how the inspection score 500 changes in connection with thedirection of change of the parameter value and the magnitude of changethereof. In an embodiment, when a more inaccurate defect inspectionresult is derived as a result of performing an operation afterconfiguring a certain value with regard to a predetermined parameter,the processor 10 may discard the corresponding value and may revert tothe parameter value based on the previous configuration.

In an embodiment, the processor 10 may find an optimal parameter valuewhile successively changing a parameter value to be applied to apredetermined parameter. When the discrete method is used, the processor10 may change the parameter value by a predetermined gradient; and, whenthe continuous method is used, the processor 10 may continuously changethe parameter value. In addition, when the optimal parameter value of aparameter is determined, the processor 10 may keep determining theoptimal parameter of the next parameter one after another.

In an embodiment, the processor 10 may determine a subset of parametersincluding a plurality of parameters among all parameters and may find anoptimal parameter value while continuously changing a parameter value tobe applied to the subset of parameters at once. Initially, the processor10 may apply predetermined parameter values to the subset of parameters.The processor 10 may perform an operation to which pre-determinedparameter values have been applied and may obtain a resulting inspectionscore. When it is determined that application of new parameter values isnecessary, the processor 10 may change all parameter values to beapplied to the subset of parameters. In this case, the value of otherparameters that are not included in the subset of parameters may befixed. After optimal parameter values of the corresponding subset ofparameters are determined, the processor 10 may determine a differentsubset of parameters and may change parameter values of the determinedsubset of parameter at once, thereby conducting optimization.

In an embodiment, the processor 10 may find an optimal parameter valuewhile keeping randomly selecting a parameter value to be applied to apredetermined parameter. The processor 10 may perform an operation usingthe parameter based on the randomly selected parameter value and maydetermine whether the random parameter value is an optimal parametervalue according to the result.

The processor 10 may determine the above-mentioned gradient value. In anembodiment, the processor 10 may determine the appropriate direction ofchange of a parameter and the appropriate magnitude of change thereofwith regard to a subset of the entire data set and may apply thedetermined direction of change and magnitude of change to the entiredata set, thereby determining the above-mentioned gradient value. Asused herein, the entire data set may refer to all of a plurality of setsof images, and the subset of the data set may refer to some images ofthe plurality of sets of images. Initially, the processor 10 mayevaluate what result is obtained when an operation based on a changedparameter is applied to a partial subset of the entire data set whilecontinuously changing a parameter value of a predetermined imageprocess. The processor 10 may determine what direction of change andwhat magnitude of change derive the optimal result through this process.When an appropriate direction of change and an appropriate magnitude ofchange are determined with regard to the data subset, the processor 10may apply the same to the entire data set, thereby determining again theappropriateness of the direction of change and the magnitude of change.The processor 10 may determine an appropriate direction of change of theparameter value and the appropriate magnitude of change thereof withregard to the entire data set through this process, and may determinethe above-mentioned gradient on the basis thereof. In an embodiment, thesubset of the entire data set may be determined randomly. In anembodiment, the processor 10 may determine the gradient of anappropriate parameter value while changing the value of some parameterswhile the parameter value of the remaining parameters remains fixed.

The processor 10 may use various optimization methods in order to findan optimal parameter of an image process used for an operation. Forexample, the processor 10 may use a gradient ascent/descent method,which is a primary iterative optimization algorithm for finding themaximum/minimum value of a function in a stepwise manner using agradient value at one point of the function, or an annealing method thatis efficient for a vast range in which an optimal value is to be foundas a probabilistic approach for estimating a global optimum in theentire range of a function.

However, the gradient ascent/descent method, for example, may have aproblem in that the method may find not the maximum/minimum value in theentire function range, but the local maxima or local minima. Forexample, a result value based on a specific variable value may be givenas in the illustrated graph 6010 of a function. In the illustrated graph6010, the x-axis may correspond to a specific parameter, that is, aparameter value in the present disclosure, and the y-axis may correspondto a result value based on a specific variable, that is, an inspectionscore in the present disclosure. The optimal result value in theillustrated graph 6010 may appear as the global maximum or globalminimum, and the optimal parameter value may be the parameter value ofthe global maximum or global minimum. However, when the gradientascent/descent method is used, for example, the local maxima or localminima, which is the maximum/minimum value in a specific range, may beerroneously determined as the global maximum or global minimum.

In an embodiment, in order to avoid deriving the local maxima or localminima, the processor 10 may apply a random value as the starting point,of an optimization method for finding an optimal parameter value. Thatis, optimization based on an optimization method may start to proceed ata point in the illustrated graph 6010, and the processor 10 may randomlydetermine the starting point. This makes it possible to avoid derivingthe local maxima or local minima as the result of the optimizationmethod. Optimization started at the random starting point may berepeatedly performed by a fixed number of times, or may be repeatedlyperformed until a predetermined condition is satisfied. A parametervalue that gives the optimal result, among parameter values obtained inthe repeated optimization process, may be determined as the optimalparameter value.

In an embodiment, the processor 10 may utilize a Gibbs measure in orderto avoid erroneously determining the local maxima or local minima as theglobal maxima or global minima. The Gibbs measure may be an optimizationmethod that utilizes a probabilistic distribution, which provides aprobabilistic representation of an inspection score 500 of an operationbased on a parameter value. The function of the probabilisticdistribution based on the Gibbs measure may be represented as by theillustrated equation 6020.

In the illustrated equation 6020, Θ may denote a vector of parametervalues, that, is, a set of parameter values. P(Θ) may denote a set ofparameter values, that is, a probability value of Θ in a parameterspace, β may denote a parameter of P(Θ) referred to as an inversetemperature. Z[β] may denote a normalizing value that represents the sumof probability values in the entire range of Θ, and may be referred toas a partition function. E(Θ) represents an evaluation of the result ofdefect inspection by an operation based on Θ, and may be referred to asan energy function.

The Z[β] may be difficult to process because it is associated with thesum of the very large entire parameter space, but a probabilisticdistribution may be obtained using a method such as MCMC (Monte CarloMarkov Chain). For example, the MCMC method introduced in “ChristopheAndrieu, Nando De Freitas, Arnaud Doucet and Michael I. Jordan AnIntroduction to MCMC for Machine Learning, 2003”, for example, defines aMarkov chain, and the stationary distribution of the Markov chain maycoincide with the probabilistic distribution of the Gibbs measure. Aparameter sample similar to P(Θ) may be obtained by extracting parametervalues from the distribution of the Markov chain. As the β valueincreases, which is an inverse temperature parameter, P(Θ) may graduallyconcentrate to the maxima of E(Θ). Therefore, a parameter sample of P(Θ)at a point with a large β value may be a parameter that becomes theminima, and this may be the optimal parameter that brings the optimalresult value.

For example, the parameter may switch from Θ0 to Θ1 by a randomselection. The new parameter Θ1 may reduce the value of E(Θ) comparedwith Θ0. In this case, it may be determined that the change to the newparameter Θ1 is a change in a direction toward the optimal parameter.Alternatively, it is may be determined according to a probability valuederived from the illustrated equation 6030 whether a change to the newparameter Θ1 is necessary. The illustrated equation 6030 may be easilycalculated without having to calculate the value of Z[β].

FIG. 7 illustrates a branching sub-sequence and a merging sub-sequenceaccording to an embedment of the present disclosure. As described above,a sequence of an operation based on a certain configuration may includeat least one sub-sequence, and the sub-sequence may include a branchingsub-sequence 7010 and a merging sub-sequence 7020.

In an embodiment, a sequence may include a branching sub-sequence 7010.As described above, a predetermined set of images 7030 may undergo twoor more image processes 220 and 221 in a parallel manner during anoperation. In this regard, an image process may be one of a plurality ofimage processes determined according to a configuration of theoperation. For example, a predetermined set of images 7030 that haveundergone an image process 210 may be inputted to two or more imageprocesses 220 and 221 in the next step. The set of images, afterundergoing two or more different image processes 220 and 221, mayundergo a line of image processes configured separately.

In an embodiment, a sequence may include a merging sub-sequence 7020. Asdescribed above, sets of images 7040 and 7050 that have undergonecertain image processes 230 and 231 in a parallel manner may undergo asingle identical image process 235.

In an embodiment, image processes 230 and 231 applied in a parallelmanner may be the same image process or different image processes. Evenif the image processes 230 and 231 are the same image process, sets ofimages 7060 and 7070 inputted to the image processes 230 and 231 maydifferent from each other, and it may thus be considered that the imageprocesses 230 and 231 are arranged in a parallel manner during theoperation. In an embodiment, the sets of images 7040 and 7050 may bedifferent sets of images that have undergone different processing. Setsof images 7040 and 7050 supposed to undergo a single identical imageprocess 235 may be outputted as a single set of images, and may undergothe same line of image processes later.

In an embodiment, M sets of images may be inputted and merged such thatN sets of images are outputted. In this case, M and N may be differentnatural numbers. The M sets of images may be sets of images that haveundergone different lines of image processes. The N sets of images maybe inputted to N different lines of image processes.

In an embodiment, a plurality of branching sub-sequences and a pluralityof merging sub-sequences may be arranged inside a single sequence. In anembodiment, without any restriction requiring that a mergingsub-sequence be arranged right after a branching sub-sequence, thebranching sub-sequence may be applied again to one of a set of branchingimages. In an embodiment, after merging a set of images, the mergingsub-sequence may not undergo a single identical image process, and maygenerate inspection data directly using the set of merging images.

In an embodiment, image processes included in a branching sub-sequenceand a merging sub-sequence may be image processes selected from theabove-mentioned plurality of image processes according to a certainconfiguration of the operation.

FIG. 8 illustrates an embodiment of an operation optimization methodthat may be performed by an electronic device 1 according to the presentdisclosure. Although respective steps of a method or algorithm accordingto the present disclosure are described in a successive order in the,illustrated flowchart, respective steps may also be performed in anycombinations and/or in any order according to the present disclosure.The description with reference to the present flowchart does not excludeapplying a change or modification to the method or algorithm, and doesnot mean that a certain step is necessary or preferred. In anembodiment, at least some steps may be performed in a parallel,repeated, or heuristic manner. In an embodiment, at least some steps maybe omitted, or different steps may be added.

The electronic device 1 according to the present disclosure may performan operation optimization method according to various embodiments of thepresent disclosure, in connection with processing one or more sets ofimages and thereby inspecting a defect of a target object. The operationoptimization method according to an embodiment of the present disclosuremay include a step of capturing one or more sets of images of a targetobject (S100); a step of determining a predetermined configuration of anoperation (S200); a step of performing an operation based on thepredetermined configuration with regard to one or more sets of images(S300); a step of generating inspection data from the processed one ormore sets of images (S400); a step of comparing the inspection data withfeature data and thereby generating an inspection score (S500); and/or astep of setting the predetermined configuration as an optimalconfiguration (S610) or determining a configuration to be applied afterthe predetermined configuration (S620) according to whether theinspection score satisfies a predetermined threshold score (S600).

In Step S100, the camera 40 of the electronic device 1 may capture oneor more sets of images of the target object. In Step S200, the processor10 may determine a predetermined configuration (for example, firstconfiguration) of the operation for processing one or more sets ofimages. In Step S300, the processor 10 may perform the operation basedon the predetermined configuration (for example, first configuration)that has been determined, with regard to one or more sets of images.

In Step S400, the processor 10 may generate inspection data 300regarding a defect or feature of the target object from the one or moresets of images processed according to the predetermined configuration(for example, first configuration). In Step S500, the processor 10 maycompare the inspection data with feature data 400 regarding a defect orfeature of the target object, stored in the memory, thereby generatingan inspection score 500.

In steps S600 to S620, the processor 10 may determine whether thegenerated inspection score is equal to or larger than a predeterminedthreshold score and may set the current configuration in which theinspection score 500 satisfies the predetermined threshold score, thatis, the predetermined configuration (for example, first configuration),as the optimal configuration. In an embodiment, the processor 10 maystore the predetermined configuration determined as the optimalconfiguration in the memory 20. When the inspection score 500 fails tosatisfy the predetermined threshold score, the processor 10 maydetermine a configuration (for example, second configuration) to beapplied after the current configuration.

In an embodiment of the operation optimization method according to thepresent disclosure, the step of determining a predeterminedconfiguration of the operation (S200) may include a step of selecting(determining), by the processor 10, a set of images (for example, firstset of images) to which the operation based on the predeterminedconfiguration (for example, first configuration) is to be applied, froma plurality of sets of images. In addition, Step S200 may include a stepof determining a set of image processes (for example, first set of imageprocesses) included in the operation based on the predeterminedconfiguration (for example, first configuration) among a plurality ofimage processes. In addition, Step S200 may include a step ofdetermining values of a parameter (for example, first parameter value)used for an image process of a set of image processes (for example,first set of image processes) by the processor 10. In addition, StepS200 may include a step of determining an application sequence (forexample, first sequence) of a set of image processes (for example, firstset of image processes) to be applied to a set of images (for example,first set of images) by the processor 10.

In an embodiment, the step of processing a plurality of sets of imagesaccording to the predetermined configuration (S300) may include a stepof applying a set of images processes (for example, first set of imageprocesses) having values of a parameter (for example, first parametervalue) to a set of images (for example, first set of images) andprocessing, the same by the processor 10.

In an embodiment, the step of generating an inspection score (S500) mayinclude a step of obtaining one or more flags from inspection data 300by the processor 10, the flags indicating whether the target object atleast one predetermined exterior feature or exterior defect; a step ofcomparing each of the one or more flags with feature data 400 anddetermining whether each of the one or more flags is consistent withfeature data 400; and/or a step of generating an inspection score thatindicates the ratio of the number of flags that are consistent with thefeature data 400 with regard to the entire number of the one or moreflags.

In an embodiment, the step of generating an inspection score (S500) mayinclude a step of obtaining an area (for example, first area), in whichat least one predetermined exterior feature or exterior defect of thetarget object exists, from the inspection data 300 by the processor 10;a step of obtaining an area (for example, first area), in which at leastone exterior feature or exterior defect of the target object exists,front the feature data 400; a step of determining an overlapping areabetween the two areas (for example, first and second areas); and/or astep of generating an inspection score that indicates the proportion ofthe overlapping area against an area (for example, second area) in whicha defect or feature indicated by the feature data 400 exists.

In an embodiment, the step of determining the next configuration (forexample, second configuration) of the operation (S620) may include astep of determining, for example, by the processor 10, a set of images(for example, second set of images) of the next configuration among aplurality of sets of images; determining a set of image processes (forexample, second set of image processes) of the next configuration amonga plurality of image processes; determining values of a parameter (forexample, second parameter value) to be used for a set of image processes(for example, second set of image processes) of the next configuration;and determining a sequence (for example, second sequence) in which a setof image processes (for example, second set of image processes) of thenext configuration are to be applied. The processor 10 may determine thenext configuration (for example, second configuration) of the operationbased on a difference between an inspection score based on apredetermined configuration (for example, first configuration) and theabove-mentioned predetermined threshold score.

In an embodiment, the step of determining the next configuration (forexample, second configuration) of the operation (S620) may include astep of deriving a difference between an inspection score generatedusing a configuration that has been applied to the operation prior to apredetermined configuration (for example, first configuration) and aninspection score generated using a predetermined configuration (forexample, first configuration). In addition, Step S620 may include a stepof modifying, for example, by the processor 10, the predeterminedconfiguration (for example, first configuration) on the basis of thederived difference and thereby determining the next configuration (forexample, second configuration).

In an embodiment, the step of determining the next configuration (forexample, second configuration) of the operation (S620) may include astep of performing an operation based on the next configuration (forexample, second configuration), for example, by the processor 10. Thatis, Step S620 may include a step of applying, for example, by theprocessor 10, a set of image processes (for example, second set of imageprocesses) having values of a parameter (for example, second parametervalue) to a set of images (for example, second set of images) accordingto an application sequence (for example, second sequence) and therebyprocessing the same.

In an embodiment, one or more sets of images of the target object may bethose captured under one or more predetermined settings. In anembodiment, the one or more predetermined settings may include, forexample, the type, amount, and arrangement of the light source 30, thetype, amount, and arrangement of the camera 40, parameters related tothe tight source 30, parameters related to the camera 40, the wavelengthof light, the illumination of light, the angle at which light isirradiated to the target object and/or any combinations thereof.

In an embodiment, a sequence (for example, first configuration) based ona predetermined configuration (for example, first configuration) mayinclude a sub-sequence for parallel application of at least twodifferent image processes among a set of image processes (for example,first set of image processes) to a set of images (for example, first setof images).

In an embodiment, a sequence (for example, first configuration) based ona predetermined configuration (for example, first configuration) mayinclude a sub-sequence for application of a single image process to aset of image processes (for example, first set of images) to which atleast two different images processes among a set of image processes (forexample, first set of image processes) have been applied in a parallelmanner.

Various embodiments of the present disclosure may be implemented assoftware in a machine-readable storage medium. The software may besoftware for implementing various embodiments of the present disclosure.The software may be inferred from various embodiments of the presentdisclosure by programmers in the technical field to which the presentdisclosure pertains. For example, the software may be a machine-readableprogram including an instruction (for example, code or code segment).The machine may be a device capable of operating according to aretrieved instruction, for example, a computer. In an embodiment, themachine may be an electronic device according to embodiments of thepresent disclosure. In an embodiment, the processor of the machine mayexecute a retrieved instruction such that elements of the machineperform a function corresponding to the instruction. In an embodiment,the processor may be a processor 10 according to embodiments of thepresent disclosure. The storage medium may refer to any type ofmachine-readable recording medium that stores data. The storage mediummay include, for example, a ROM, a RAM, a CD-ROM, a magnetic tape, afloppy disk, an optical data storage device and the like. In anembodiment, the storage medium may be a memory 20. In an embodiment, thestorage medium may be implemented so as to be distributed over acomputer system connected through a network or the like. The softwaremay be distributed over a computer system or the like, stored therein,and performed accordingly. The storage medium may be a non-transitorystorage medium. The non-transitory storage medium refers to a tangiblemedium regardless of whether data is stored semi-permanently ortemporarily, and does not include any signal that propagates in atransitory manner.

Although the technical idea of the present disclosure has been describedabove with reference to various embodiments, the technical idea of thepresent disclosure includes various substitutions, modifications, andchanges that could be made within a scope that could be understood by aperson skilled in the art to which the present disclosure pertains.Furthermore, such substitutions, modifications, and changes are to beunderstood as being included in the accompanying claims.

MODE FOR THE INVENTION

Various embodiments of the present disclosure have been described in theBest Mode.

INDUSTRIAL APPLICABILITY

The present disclosure is applicable to a field related to inspection ofa target object.

1. An apparatus for optimizing an inspection of an exterior of a targetobject, comprising: a light source configured to irradiate light to thetarget object; at least one camera configured to capture one or moresets of images of the target object, each set of images being capturedunder one or more predetermined settings; a memory configured to storethe one or more sets of images of the target object and feature dataincluding one or more predetermined exterior features of the targetobject; and a processor configured to: obtain the one or more sets ofimages from the memory; determine a first configuration of an operationfor processing the one or more sets of images to inspect the targetobject, wherein the first configuration includes at least one of a firstset of images on which the operation is to be performed among the one ormore sets of images, a first set of image processes to be performed inthe operation among a plurality of image processes, first parametervalues for the first set of image processes, and a first sequence thateach image process of the first set of image processes is performed inthe operation; perform the operation on the first set of images underthe first configuration; generate inspection data on the exterior of thetarget object from the first set of images that has been processed underthe first configuration; generate an inspection score that indicates aproportion in pixels of an overlapping area between a first area on thetarget object from the inspection data and a second area on the targetobject from the feature data with respect to the second area; comparethe inspection score with a predetermined threshold score; set the firstconfiguration as an optimal configuration of the operation if theinspection score satisfies the predetermined threshold score; determinea second configuration of the operation by modifying the firstconfiguration if the inspection score does not satisfy the predeterminedthreshold score; and control the light source and the at least onecamera to obtain additional sets of images of the target object insettings that are different from the one or more predetermined settings,when inspection scores do not satisfy the predetermined threshold scoreafter repeating the modification of configurations of the operation apredetermined number of times.
 2. The apparatus of claim 1, wherein theprocessor, when generating the inspection score, is further configuredto: obtain one or more flags from the inspection data, each of the oneor more flags indicating whether the exterior of the target objectincludes at least one of the predetermined exterior features; determinewhether each of the one or more flags is consistent with the featuredata stored in the memory; and generate the inspection score thatindicates a ratio of a number of flags that are consistent with thefeature data to a total number of the one or more flags.
 3. Theapparatus of claim 1, wherein the processor, when generating theinspection score, is further configured to: obtain the first area on thetarget object from the inspection data, the first area including atleast one of the predetermined exterior features; obtain the second areaon the target object from the feature data pre-stored in the memory, thesecond area including the at least one of the predetermined exteriorfeatures; determine the overlapping area between the first area and thesecond area; and generate the inspection score that indicates theproportion of the overlapping area with respect to the second area. 4.The apparatus of claim 1, wherein the processor, when determining thesecond configuration, is further configured to: determine at least oneof a second set of images from the one or more sets of images, a secondset of image processes from the plurality of image processes, secondparameter values for the second set of image processes, and a secondsequence of the second set of image processes, according to a differencebetween the inspection score and the predetermined threshold score. 5.The apparatus of claim 1, wherein the processor, when determining thesecond configuration, is further configured to: derive a differencebetween a previous inspection score, which has been generated using aprevious configuration of the operation before performing the operationunder the first configuration, and the inspection score generated usingthe first configuration of the operation; and determine the secondconfiguration by modifying the first configuration according to thederived difference.
 6. The apparatus of claim 4, wherein the processoris further configured to perform the operation under the secondconfiguration.
 7. The apparatus of claim 1, wherein the one or morepredetermined settings include at least one of a wavelength of the lightirradiated to the target object, an irradiation angle of the light intothe target object, an illumination of the light, and a type of the atleast one camera.
 8. The apparatus of claim 1, wherein the firstsequence includes a sub-sequence that applies at least two differentimage processes, among the first set of image processes, to the firstset of images in parallel.
 9. The apparatus of claim 8, wherein thefirst sequence includes a sub-sequence that applies a single imageprocess, among the first set of image processes, to the first set ofimages after applying the at least two different image processes to thefirst set of images in parallel.
 10. A method for optimizing aninspection of an exterior of a target object, comprising: irradiating,by a light source, light to the target object; capturing, by at leastone camera, one or more sets of images of the target object, each set ofimages being captured under one or more predetermined settings;determining a first configuration of an operation for processing the oneor more sets of images to inspect the target object, wherein the firstconfiguration includes at least one of a first set of images on whichthe operation is to be performed among the one or more sets of images, afirst set of image processes to be performed in the operation among aplurality of image processes, first parameter values for the first setof image processes, and a first sequence that each image process of thefirst set of image processes is performed in the operation; performingthe operation on the first set of images under the first configuration;generating inspection data on the exterior of the target object from thefirst set of images that has been processed under the firstconfiguration; generating an inspection score that indicates aproportion in pixels of an overlapping area between a first area on thetarget object from the inspection data and a second area on the targetobject from feature data with respect to the second area, the featuredata being stored in a memory and including one or more predeterminedexterior features of the target object; comparing the inspection scorewith a predetermined threshold score; setting the first configuration asan optimal configuration of the operation if the inspection scoresatisfies the predetermined threshold score; determining a secondconfiguration of the operation by modifying the first configuration ifthe inspection score does not satisfy the predetermined threshold score;and controlling the light source and the at least one camera to obtainadditional sets of images of the target object in settings that aredifferent from the one or more predetermined settings, when inspectionscores do not satisfy the predetermined threshold score after repeatingthe modification of configurations of the operation a predeterminednumber of times.
 11. The method of claim 10, wherein the generating theinspection score includes: obtaining one or more flags from theinspection data, each of the one or more flags indicating whether theexterior of the target object includes at least one of the predeterminedexterior features; determining whether each of the one or more flags isconsistent with the feature data pre-stored in the memory; andgenerating the inspection score that indicates a ratio of a number offlags that are consistent with the feature data to a total number of theone or more flags.
 12. The method of claim 10, wherein the generatingthe inspection score includes: obtaining the first area on the targetobject from the inspection data, the first area including at least oneof the predetermined exterior features; obtaining the second area on thetarget object from the feature data pre-stored in the memory, the secondarea including the at least one of the predetermined exterior features;determining the overlapping area between the first area and the secondarea; and generating the inspection score that indicates the proportionof the overlapping area with respect to the second area.
 13. The methodof claim 10, wherein the determining the second configuration for theoperation includes determining at least one of a second set of imagesfrom the one or more sets of images, a second set of image processesfrom the plurality of image processes, second parameter values for thesecond set of image processes, and a second sequence of the second setof image processes, according to a difference between the inspectionscore and the predetermined threshold score.
 14. The method of claim 10,wherein the determining the second configuration for the operationincludes: deriving a difference between a previous inspection score,which has been generated using a previous configuration of the operationbefore performing the operation under the first configuration, and theinspection score generated using the first configuration of theoperation; and determining the second configuration by modifying thefirst configuration according to the derived difference.
 15. The methodof claim 13, further comprising performing the operation under thesecond configuration.
 16. The method of claim 10, wherein the one ormore predetermined settings include at least one of a wavelength oflight irradiated to the target object for capturing the one or more setsof images of the target object, an irradiation angle of the light intothe target object, an illumination of the light, and a type of at leastone camera configured to capture the one or more sets of images of thetarget object.
 17. The method of claim 10, wherein the first sequenceincludes a sub-sequence that applies at least two different imageprocesses, among the first set of image processes, to the first set ofimages in parallel.
 18. The method of claim 17, wherein the firstsequence includes a sub-sequence that applies a single image process,among the first set of image processes, to the first set of images afterapplying the at least two different image processes to the first set ofimages in parallel.